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A study tested the notion that information stories in 
"USA Today" are more readable because its editors shorten them by 
leaving out background details. It also examined whether readers 
comprehend as much from stories written in "USA Today" style as they 
do from stories written in the more traditional "New York Times" 
style. For the Readability Study seven readability formulas were 
applied to two stories reported in both papers. For the Reader 
Comprehension Study, 166 undergraduate subjects read either the "New 
York Times" or the "USA Today" versions of the stories. Results of 
the readability measures showed that both versions of the stories 
were at or above college level, but only the Fog index significantly 
discriminated between the versions of the stories, finding the "USA 
Today" version more readable in terms of sentence length and 
complexity. The comprehension study results showed that readers of 
the "USA Today" stories comprehend significantly less than do readers 
of the same stories in "New York Times" style. In contrast to 
arguments of "USA Today" editors, multiple regressions showed that 
newspaper use was negatively related to comprehension of "USA Today" 
stories. However, frequent and attentive reading of "hard news" 
stories is a predictor of better comprehension for "USA Today" 
stories, while "soft news" stories were related to better 
comprehension for "New York Times" stories. Findings suggest that 
"USA Today" editors may be placing reader comprehension at risk when 
they shorten analysis pieces by leaving out background. (Three tables 
of data are included; 19 references are attached.) (Author/RS) 
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Does Shorter Mean Easier to Understand? 
A Study of Comprehension of USA Today Information Stories. 



This study was designed to see if USA Today information stories are more 
readable and more comprehensible. Readability analyses showed that stories 
from the USA Today and the New York Times were both at college level, but USA 
Today stories were more readable. However, readers of USA Today stories 
comprehended less, and frequent newspaper use was negatively related to 
comprehension. The authors argue that USA Today editors may risking reader 
comprehension when they shorten stories. 
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Does Shorter Mean Easier to Understand? 
A Study of Comprehension of USA Today Information Stories. 



This study was designed to test the notion that information stories in USA Today are 
more readable because its editors shorten them by leaving out background details. The 
study also examined whether readers comprehend as much from stories written in USA 
Today style as they do from stories written in the more traditional, complete New York 
Times style. Seven readability formulas were applied to two stories reported in both 
papers. In addition, 166 subjects read either the New York Times or the USA Today 
versions of the stories. Results of the readability measures showed that both versions 
of the stories were at or above college level, but only the Fog index significantly 
discriminated between the versions of the stories, finding the USA Today version more 
readable in terms of sentence length and complexity. The comprehension study results 
showed that readers of USA Today stories comprehended significantly less than did 
readers of the same stories in New York Times style. In contrast to arguments of USA 
Today editors, who say that their readership doesn't need background because they use 
USA Today as a supplement to other newspaper reading, multiple regressions showed 
that newspaper use was negatively related to comprehension of USA Today stories. 
However, frequent and attentive reading of "hard news" stories is a predictor of better 
comprehension for USA Today stones, while "soft news" stories were related to better 
comprehension for New York Times stories. The authors argue that USA Today editors 
may be placing reader comprehension at risk when they shorten analysis pieces by 
leaving out background. 
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Does Shorter Mean Easier to Understand? 
A Study of Comprehension of USA Today Information Stories. 

Renee A. Botta, Suzanne Pingree and Robert P. Hawkins 
University of Wisconsin-Madison 

How best to present the news? What do readers need in order to comprehend the 
issues of the day? How much complexity do they want, or will they put up with? Do 
simplicity of vocabulary and grammar matter? Can editors assume background 
knowledge of issues, or must this be provided as part of stories? These and similar 
issues have been widely discussed by journalists (and less often by researchers) for 
years, and have been at the heart of many debates over reshaping newspapers around 
the country to respond to perceived changes in the needs and desires of readers. 

In many ways, the most radical model for such "remakes" has been USA Today, 
setting a benchmark with shorter stories, graphics, larger type, and in other ways 
supposedly making the paper easier to digest than a typical daily. ^ Implicit (and 
sometimes explicit) in reactions to this style is the notion that the presentation style of 
USA Today is substituting surface glitter for substance, and that efforts to attract the 
audience are at the expense of providing the sort of information that readers need to 

^George A. Gladney, "The McPaper Revolution? USA Today-Style Innovation at large U.S. Dailies," 
Newspaper Research Journal. l3:S4~7irWinter/Spring, 1992). 
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really understand the news. USA Today management has argued instead that it 
purposely removes background details for brevity because its readers are well-informed 
newspaper consumers, typically reading more than one paper daily, so that USA Today 
functions largely as a supplement, a second paper.^ 

Whether this is true or not as a ger)ej?alization, the essential questions remain: How 
has USA Today SKTiplified its stories? Are they shorter, more readable, missing minor 
facts or some combination of the three? More importantly, what happens when readers 
come to these simplified stories? For example, does lack of background information 
make comprehension of the essentials easier because they are "stripped down," or is it 
instead more difficult to understand the essentials without the background? 

Some critics argue that USA Today's style encourages brevity - shorter sentences, 
shorter stories and easier-to-digest text. Gladney^ found that USA Today's mean length 
for lead sentences was 21 .5 words, compared to several large dailies with a mean of 27.6 
words. If leads are any indication of story length, then USA Today stories would seem 
to be shorter. But are these (possibly) shorter stories easier to read, too? There is some 
reason to believe that this would be true. Catalano"* argued that longer sentences 
"incorporate more words, and more words mean more relationships, which increase the 

^Peter Prichard. The Making of McPaper: The Inside Storv of USA Today (Kansas City: Andrews, McMeel 
& Parker, 1987). 

Gladney, op. cit. 

''Kevin Catalano, "On the Wire: How Six News Services are Exceeding Readability Standards," Journalism 
Ouarterlv. 67:1:97-103 (Spring, 1990). 
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effort for the reader,,. Long, complex sentences are barriers to comprehension," 

However, one could also argue that shorter sentences means less information. If 
the information that is not provided would have offered important background details, 
then the brevity of USA Today stories might actually interfere with reader comprehension. 
That is, if the missing information contains useful causal connections, bridges, transitions, 
or ways to help the reader make links between ideas, then reader comprehension would 
decrease,^ And increased background information, whether within the story or as a 
sidebar, has been found to increase comprehension.^ 

This study was designed to test some of these ideas in two ways. First, we 
wanted to explore further the notion that USA Today is easy to digest by evaluating 
stories according to a number of measures of readability. We have some evidence that 
USA Today stories are shorter, but none that they are actually easier to read. 

Second, we wanted to pursue some of the contradictory commonsense ideas 
about reader comprehension of shorter stories with missing details. On the one hand, 
some research implies that readers comprehend less when background and other 
explanatory information are removed. On the other hand, USA Today argues that 



See, for example: Arthur C. Graesser, S.P. Robertson and P.A. Anderson, "Incorporating Inferences in 
Narrative Representations: A study of how and why," Co gnitive Psychology . 13:1:1-26, (1981); Susan Kemper, 
"Measuring the Inference Load of Text," Journal of Educational Psycholog y. 75:3:391-401, (June, 1983); Susan 
Kemper, "Filling in the Missing Links," Journal of Verbal Learning and Verbal Behavion 21:1:99-107, (1982). 

^Jeffrey L. Griffin and Robert L. Stevenson, "Influence of Text and Graphics in Increasing Understanding 
of Foreign News Context," Newspaper Research Journal . 13:84-99, (Winter/Spring, 1992); Douglas B. Ward, "The 
Effectiveness of Sidebar Graphics," Journalism Quarterly . 69:2:318-328, (Summer, 1992). 
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removing this information will not affect comprehension for a well-informed reader. 

Third, we were interested in how readers evaluate fairly complete vs. shorter, easier 
to read stories with information missing. Do they have a standard of journalistic quality 
that is applied to their reading? How aware are they of the differences between USA 
Today and, for example, the New York Times, and how do those evaluations affect their 
comprehension? 

METHODS 

Readability Study 

We selected two fairly complex stories from the front page of issues of USA Today 
during 1988.^ One story was about the state of education in the U.S. today, and the 
other story was about a Supreme Court ruling on digging in people's trash. Because we 
wanted to be able to compare the readability of USA Today stories against a 
comprehensive standard, we selected stories on the same subjects from the front page 
of the New York Times, a newspaper well-known for its quality coverage.^ 

We used seven different readability tests. The Dale-Chall readability formula rates 
material above fifth grade level based on the number of words in the selection that do not 



USA Today . Jan. 1 through June 30, 1988. 
'New York Times . Jan. 1 through June 30, 1988. 
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appear on the Dale list of 3,000 words, "known by at least 80 percent of the children in 
grade four."^ (That this list of words was developed in 1 948 makes it at least somewhat 
problematic for today's language.) The Spache formula is also based on a word list, not 
quite as out of date (1974). However, it too is problematic for this study because it tops 
out at the 4th grade reading level.^° We expected that the stories from both 
newspapers would be above that level. 

The Fry readability formula rates material from first grade through college levels of 
difficulty based on the number of syllables in the sample and the number of words and 
sentences.^^ The Flesch readability formula rates material on a score from 0 to 100 with 
0 being extremely easy and 1 00 being very difficult. The Flesch uses number of syllables, 
words and sentences, like the Fry, but it also describes typical reading for each level 
(very difficult to very easy).^^ The Raygor formula rates third grade through college and 
beyond based on the number of words with six or more letters and the number of words 
and sentences in the sample. ^-^ The Fog formula rates for first grade and higher based 
on the number of words with three or more syllables and the number of words and 

^Edgar Dale and Jeanne S. Chall, Formula for Predicting Readability (Ohio: Bureau of Educational research, 
1948). 

^^'George D. Spache, Good Reading for Poor Readers . (Champaign, 111.: Garrard Pub. Co., 1974). 
^^Edward B. Fry, Fry Readability scale: Extended. (Providence: Jamestown Publishers, 1978). 
^^Rudolf Flesch, How to Test Readability . (New York: Harper, 1951). 

• Alton L. Raygor, Effective Reading:Improvin£ Reading Rates and Comprehension. (New York: McGraw- 
Hiil, 1985). 
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sentences in the sample.^"* Finally, the SMOG formula rates for third grade and higher 
based on the number of words with three or more syllables and the number of words and 
sentences in the sample. The SMOG formula was applied to samples of the four stories, 
since it is only intended to be used on samples of 30 sentences. 

We used all seven formulas in an effort to be exhaustive, and to discover as much 
as we could both about the readability of the stories and the sensitivity of the various 
readability formulas. 

Reader Comprehension Study 

A questionnaire was administered to 166 undergraduates in various courses (not 
journalism) at a large Midwestern university. The subjects were randomly assigned to the 
USA Today stories or the New York Times stories used in the readability study. 

The stories were prepared so that information that would identify their source was 
removed. The students were told these were actual stories that had appeared in a 
newspaper, and were asked to read them as they would a news article. 

The students completed some media use items and demographics, then read the 
nxi^ws stories (two New York Times stories or two USA Today stories), then completed 

^"^Robert Gunning, The Technique of Clear Writing . (New York: McGraw-Hill, 1968). 

^^G. Harry McLaughlin, "SMOG grading - a new readability formula," Journal of Reading . 12:639-646. 

^^William C. Porter, "The Value of Readability Studies," Editor & Publisher . May, 1982, p. 84; Ron Smith, 
"How Consistently do Readability Tests Measure the Difficulty of Newswriting?" Newspaper RCvSearch Journal , 
5:4, (Summer, 1984). 
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comprehension and evaluation measures. 

Newspaper Use Measures. 

The first section of the questionnaire consisted of questions about newspaper use: 
How many days a week do you read the newspaper, how much time 'a day do you 
spend reading the newspaper, how often do you read certain kinds of stories (followed 
by a list of story types), and how much attention do you pay to those kinds of stories 
(again followed by a list)? 

We began by constructing five measures of newspaper use: a measure of minutes 
per week of total newspaper use (calculated from their answers to the first two 
questions), a frequency measure for time spent with "hard news" stories (international 
news, national government news, local government news), a summary attention measure 
for hard news stories, and two similar measures for soft news stories (stories affection the 
family, feature stories, entertainment and sports stories). However, frequency and 
attention measures for hard news, and frequency and attention measures for soft news 
were highly correlated (.85 for the two hard news measures, and .68 for the two soft news 
measures). Therefore, we summed the two hard news measures to create a single 
measure of hai d news use, and did the same for soft news use. Our final newspaper use 
variables were minutes per week, hard news use and soft news use. These measures 
were significantly related to each other (range of correlations was .35 to .5), but we left 



them as separate indices because they tap different conceptual indicators of newspaper 
use. 

Story Comprehension 

Immediately after the students read the stories, they were asked to paraphrase 
what they had just read in as much detail as possible. We used an open-ended question 
instead of a set of multiple choice items since some research suggests that open-ended 
questions are a better way to test comprehension.^^ 

Each student's paraphrasing of the stories was scored according to a list of facts 
that were common to both versions of the {New York Times, USA Today) story. The fact 
list was used as a checklist against which facts present in the paraphrasing was scored. 
Scoring was done blind to the student's condition. 

Story Evaluation 

After paraphrasing the story, the students were asked to evaluate the stories they 
had read on Likert scales. These evaluations included; in depth, entertaining, reliable, 
complicated, well-written, told me all i needed to know, gave enough background 

See, for example: Veda Charrow, "Readability vs. Comprehensibility: A Case Study in Improving a Real 
Document," in Alice Davison and Georgia Green, eds.. Linguistic Complexity and Text Comprehension: 
Readability Issues Reconsidered, (New Jersey: Lawrence Erlbaum Associates, 1988), pp. 85-114; Georgia M. 
Green and Margaret Olson, "Preferences for and Comprehension of Original and Readability-Adapted 
Materials," in Alice Davison and Georgia Green, eds., Linguistic Complexity and Text Comprehension: 
Re adability Issues Reconsidered, (New Jersey: Lawrence Erlbaum Associates, 1988), pp. 115-140. 
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information, biased, informative, interesting, did not take too long to read, made me want 
to find our more about it, boring, I did not want to finish the story, it told me both sides 
of the story, and it was easy to understand. We factor analyzed these items and came 
up with a three-factor solution (see Table 1), naming the three factors "surface evaluation," 
"thoughtful critique," and "turned oft." This three-factor solution has some multiple 
loadings among variables, but it made sufficient sense conceptually that we decided to 
use it in the analysis. We constructed the three measures using factor scores. 



TABLE 1 ABOUT HERE 



RESULTS 

Readability Study. 

Table 2 shows the results of our readability analyses, with results for all of the tests 
except the Spache, which is not included in the table because both newspapers scored 
above its top level, in general, the various readability formulas show very high scores -- 
mostly college levels - for all tests, and little difference between condition. Only one of 
the formulas (the Fog) discriminated between the two versions of the stories, with the 
New York Times stories both significantly (t=4.1 , p<.05) more difficult than the USA Today 
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stories. 



TABLE 2 ABOUT HERE 



Reader Comprehension Study. 

Overall, readers who read science stories from USA Today had lower 
comprehension scores (mean = 3.32) than did readers who read the New York Times 
versions of the stories (mean = 4.07), and the difference was significant (t = 2.20, g 
=.03), This would lend support to the idea that the USA Today stories attain brevity (and 
possibly greater readability), at least according to the Fog) by way of being less 
informative. .Apparently, readers do not get as much information out of them as they do 
out of the more complete New York Times stories, even though the comprehension 
measure is based on facts present in both stories. 

But USA Today editors would argue that this is not a fair test, given their argument 
that the newspaper is a supplement to other newspapers for their well-informed 
readership. Readers who have a stronger newspaper habit should have enough general 
background knowledge to fill in the gaps in USA Today stories and be as good at 
comprehension as readers seeing more complete New York Times stories. Thus, we 
need to do an analysis that examines the effect of newspaper use on comprehension. 
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Also, we wanted to explore the extent to which readers felt that something was lacking 
in the USA Today stories, a feeling that we thought might be captured in our evaluation 
measures of the stories. Therefore, we decided to look at comprehension again in an 
analysis where newspaper use and evaluations of the stories were comprehension 
predictors. We ran a multiple regression with demographics (age and gender) entered 
first, newspaper use and attention as a second block, and evaluations of the stories as 
the third block. The results of this analysis are presented in Table 3. 



TABLE 3 ABOUT HERE 



As Table 3 shows, neither gender nor age (within the attenuated range of a student 
sample) was a significant predictor of comprehension for either the USA Today or the 
New York Times version. While both blocks for past newspaper use predicted 
comprehension sigriificaotly^for both versions of the stories, the nature of those 
predictions was different. For the New York Times, those who reported more often 
reading and paying greater attention to "soft" news comprehended more of these stories, 
while readership of "hard" news was marginally negatively related to comprehension. 

The picture for USA Today is completely different and has mixed implications for the 
claims that its readers do not need full background because of their greater ongoing 
experience with other papers. Frequency of typical newspaper readership was related 
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to comprehending the USA Toc/ay stories, but it was less frequent newspaper reading that 
was associated with greater comprehension. On the other hand, and in contrast to the 
New York Times, frequency of reading and attending to "hard" news was associated with 
greater comprehension of these stories. Reading and attending to "soft" news made no 
contribution to comprehension. 

Finally, one of the three dimensions of evaluations of the stories predicted additional 
variance. While the two critique dimensions did not affect comprehension, being turned 
off by the story was associated with lessened comprehension for both versions. 

DISCUSSION 

The results of our readability study suggest that both versions of the stories were 
easy to read ONLY for college educated audiences and above. One readability formula 
(the Fog) discriminated between the New York Times and the USA Today versions of the 
same story. The Fog, as did all the other readability formulas, showed that both stories 
were both written at a fairly high level. But the Fog found that USA Today stories were 
at a significantly lower readability level than were the New York Times stories (t=4.1, 
p<.05), suggesting that the USA Today stories had fewer three syllable words and 
somewhat shorter sentences. 

As noted earlier, part of the appeal of USA Today lies in its brevity and its supposed 
ease of reading for a broad but sophisticated audience pressed for time. Criticisms of 
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the paper as merely a "headline service" have been countered by claims that what is 
omitted is the background not needed by knowledgeable readers using the paper as a 
supplement. 

Comparing comprehension of two USA Today stories with comprehension of two 
stories on the same topic taken from the New York Times {generally regarded as at the 
opposite pole in providing background information) thus provided an opportunity to test 
these claims. Overall, respondents include somewhat fewer target facts {present in both 
versions) in their paraphrases after reading the USA Today version than after reading the 
longer New York Times version, suggesting that lack of background information does 
matter. But this overall difference does not really test the claims made. It is possible that 
USA Today stories produce lower comprehension overall while showing no decrement 
from longer versions for the subgroup of readers for whom it was designed - those who 
use it as a supplement to regular reading of other newspapers. 

The multiple regressions of Table 3, run separately for the two versions, provide a 
more direct test of this idea. Frequency of newspaper reading was related to 
comprehension of USA Today in the opposite direction: greater learning for less-frequent 
readers than for regular readers. Given the overall difference in comprehension between 
the versions, the coefficients suggest that frequency of newspaper reading makes little 
difference in comprehension of longer-form stories, such as those found in the New York 
Times. But for the shorter stories of USA Today, the lower overall comprehension comes 
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not from inexperienced readers (not the target audience), but from the more frequent and 
experience newspaper readers for whom the paper was designed. 

The picture is more complicated than this, however, in that reading ("how often") and 
attention to ("how much") hard news is independently associated with greater 
comprehension of the shorter USA Today stories. That is, for a particular type of 
newspaper content, hard news, the USA Today argument does seem to hold. These two 
seemingly contradictory results are not regression artifacts (i.e., they reflect same- 
direction zero-order relationships), and must both be explained. To effectively 
comprehend USA Today stories lacking in background information, simple frequency of 
newspaper reading is not enough. Apparently, readership itself confers no special skills 
for comprehending newspapers in general (witness the same-sign but non-significant 
relationship for the New York Times stories), and in fact may instead produce an 
expectation of greater background information. When this expectation is not met, 
comprehension suffers, whereas infrequent readers do relatively better. However, 
frequent and attentive reading of hard news stories (themselves more likely to be written 
tersely and in inverted-pyramid style than soft news stories) seems to provide the training 
and orientation necessary to cope with USA Today stories. 

For the fuller New York Times stories, the role of newspaper experience is quite 
different. There is little impact of simple amount of newspaper readership, but reading 
and attending to hard and soft news are oppositely related to comprehension. We think 
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the key issue here is not the typical content meaning of the labels hard and soft, but 
rather the forms that each also refers to. Reading hard news stories (more likely to be 
constrained by traditional news style considerations - often shorter, fact-loaded inverted 
pyramids) does not prepare one for comprehending long-form analyses, and those who 
read and attend more to hard news in fact learned less from these long analytic stories. 
However, soft news stories are usually long, follow different narrative structures in which 
facts are dispersed and background is provided. And reading such stories, even if the 
content is often people-oriented and non-analytic, prepares one to read analysis in the 
same format. 

We must also consider an alternative explanation. Even though the comprehension 
measure counted facts present in both stories, it is possible that the shortness of the USA 
Today stories produced a bias in respondents' reproductions of the stories. That is, if 
readers expect that some proportion of the newspaper story to be background 
information, or that they should exercise some selection, they will reproduce a shorter 
version containing fewer essential facts after reading a shorter story. Thus, it is still 
possible that readers of USA Today stories learned as much as those who read the 
corresponding story from the New York Times. However, even this explanation is 
problematic, since it suggests a devaluing and potential disregarding of essential 
information presented in this brief format. 

We regard this alternate explanation is convoluted and unlikely, given the instructions 
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to respondents to paraphrase in as much detail as possible. A more likely bias would 
be for the inclusion of incidental or background information in paraphrases of New York 
Times stories, with the potential to leave out some of the target facts through simple 
overload. All in all, we think it is much more plausible to regard these scores, and 
differences in them, as reflecting comprehension differences stemming from these two 
presentation styles. 

The research reported here is limited to a particular type of story, analytic 
presentation of "issue" news. As such, its generalization to incident news or to people 
features is uncertain and deserves further exploration. But for this important type of 
news, the implications of story format are both more striking and more important than we 
anticipated. By writing analysis pieces without background in what resembles traditional 
hard/incident format, USA Today does place comprehension at risk. Simple frequency 
of newspaper readership is not enough to equip one to cope with this combination, thus 
calling the assumptions of the paper into serious question. Instead, it is apparently 
experience with and a history of attention to the physical format of short, fact-filled stories 
that prepares one to read and comprehend USA Today and the experience with a longer- 
form, more relaxed presentation style that prepares one for identical content in New York 
Times format. 
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TABLE 1 



FACTOR ANALYSIS FOR EVALUATIONS 



- 


Surface 
Evaluation 


Thoughful 
Critique 


Turned 
Off 


Not want to finish 


.01 


.24 


.87 


Told all needed 


.30 


.58 


.38 


In depth 


.54 


.29 


.51 


Entertaining 


.81 


.23 


.22 


Interesting 


.89 


.19 


.05 


Did not take too long 


.27 


.78 


-.08 


Informative 


.35 


.74 


.20 


Enough background 


.23 


.61 


.50 


Easy to understand 


.69 


.37 


.01 


Well-written 


.69 


.45 


.18 


Reliable 


.57 


.48 


.37 


Complicated 


.21 


.37 


.68 


Told both sides 


.26 


.64 


.41 


Wanted more info 


.75 


.16 


.22 


Boring 


.19 


.00 


.84 


Biased 


.21 


.58 


.39 
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TABLE 2 



READABILHY 





School 


Trash 




USA Todav 


NY Times 

1^1 1 il 1 Iww 


USA Todav 


NY Timp^ 


Frv 
rry 


11.5 


12+ 


11.6 


12+ 


Flesch^ 


41 .5 


35.7 


43 


29.7 


Raygor 


12+ 


12+ 


12+ 


12+ 


Fog 


15* 


19.3* 


15* 


20.7* 


Smog 


13 


14.7 


13.2 


16.2 


Dale-Chall 


15 


14 


11.9 


14.7 



^(All scores are grade levels except Flesch, which is based on a 0 to 1 00 scale with 0 
being extremely easy and 1 00 being extremely difficult.) 



* p<.05 
'* p<.01 
** p<.001 
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TABLE 3 



COMPREHENSION 





USA TODAY 


NEW YORK TIMES 


Age 


.13 


.08 




.Oo 


-.09 




.03 


.04 


Minutes Per Week Newspaper 


-.21 


-.10 


Hard News 


.24 


-.22 


Soft News 


.05 


.31* 




.09* 


.11* 


Surface Critique 


.04 


.09 


Thoughtful Critique 


-.12 


.03 


Turned Off 


-.29* 


-.27* 


R2 


.10* 


.07 


TOTAL R2 


.22** 


.21** 


(N) 


(89) 


(77) 



* p<.05 
'* p<.01 
** p<.001 
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